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ABSTRACT 



The Victorian (Australia) assessment approach of 



developing national subject profiles has potential for language 
assessment. Withiu each language area, levels of development would be 
identified and def^ led by observable language behavior, which could 
then iDe tested by a variety of test types. Assessment would be done 
by teachers in schools using standard assessment tasks, then 
interpreted according to descriptions provided for each level of 
proficiency. A project to develop such a test for first language 
proficiency consisteC of workshops with classroom teachers to 
identify observable behaviors as criteria, intensive observation of 
students for validation, creation of an initial development scale, 
field testing of the scales, and establishment of norms. A language 
proficiency assessment system that similarly uses standard assessment 
tasks and common subject profile reporting could meet several 
important criteria: It would be analytical, diagnostic, and 
criterion-referenced; enable interpretation that is progressive, 
developmental, and cumulative; use consensus moderation and empirical 
calibration; be teacher-controlled and developed; be flexible; be 
reliable and valid; and describe student behavior in terms 
commuricable to parents. What is needed is considerable development 
work, careful explanation to schools and community, technical 
assistance for districts, and resources to develop, implement, and 
maintain the system. (MSE) 
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Australia is currently underaoinq a awakening to the importance of the population 
developing proficiency in languages other than English, Our trading capacity has been 
recognised as being deficient when those responsible for dealing with exporters m our 
major trading partners cannot deal with vhem in their own language. Without this 
competence our traders are at a disadvantage. But there are more than economic reasons 
being espoused for the development of second language competence among the Australian 
population. We aspire to be a multicultural societ/. Our national and state educational 
and social justice policies outline the need for tolerance, understanding and cooperation 
among groups with different cultural and language backgrounds, without access to the 
languages there is little possibility of gaining ar< understanding of the cultures and of 
blending the Australian community into a tolerant and cooperative society. This applies 
to the access of English for migrants and the access cf native English speakers to the 
languages of our major trading and immigrant groups. 

The National Policy on Languages (Lo Bianco. 1988) and the Victorian Government Languages 
Action Plan (Lo Bianco, 1989) outline che commitment at a government level to achieving 
the twin aims of developing proficiency and providing access to all to the languages and 
cultures of the major language groups of our migrants and trading partners. The policy 
and action plan clearly s^ate that every school should offer at least one language other 
than English and every student should become proficient in at least one language other 
than the mother tongue. 

"The goal for Australian schools is Bilingualism* That is proficiency in two 
languages^ not necessarily equal competence but the highest level of skill 
possible. (Lo Bianco, 1989, p. 12) 

In order to achieve this goal there are important pre conditions which include the obvious 
resources, sufficient teacherL'* appropriate curriculum, motivated school decision makers 
and a sympathetic school community. All of these are pre conditions to the introduction 
of the program and The Action Plan (Lo Bianco, 1989} outlines the general approach to 
achieving this. There is an additional need to chart the progress of individuals, of 
classes, of schools end of entire states systems towards achieving the goals of the 
national and state policies. The meeting of the State and Commonwealth ministers of 
education in 1989 have already stipulated that systems will need to gather information on 
the progress towards national goals. The thrust towards this approach to ronitoring 
systems is clearly aimed at rationalizing policies. There is no point in arguing for 
additional resources, additional time in the curriculum or for increased status if it 
cannot be demonstrated how this will reet the aims and needs of policies at state and 
national. The terms of the policies and action plan^ are clear. Schools are expected to 
provide access to hi] ingualism, defined in terms of proficiency and competence, to all 
students. Providing instruction is not enough. Arguing for additional resources to 
enable the school or the system to provide the course and instruction is not enough. 
There are uany competing for resources. There are not many ccmpetlng for the chance to 
define the learning outcomes or to demonstrate that these can make a contribution to the 
realisation of the aims of the policies. 

The notion of proficiency in language is essential to the development of the language 
curriculum. Students, classes, schools and systems all netd to demonstrate their 
progress towards ths development of proficiency. Continuous Assessment and monitoring of 
proficiency is considered to be central to the achievement of the goals of the policy 
statements. Assessment and monitoring of development of proficiency needs to be placed in 
the perspective of the national language policy and the desire of the Australian States 
and Commonwealth tc monicor and profile the development of students in all areas of 
learning. The collective ministers of education in April this year considered the options 
for national assessment. Support for subjifct profiles* records of achievement and 
continuous monitoring wae established. At the Australian Education Council, there was 
considerable support for the development of student profiles. Several a priori conditions 
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were established. First there had to be a close relationship between curricuU'i and 
assessment; the conplexity of achievements should be reflected in assessments; and 
assessments should be criterion based. 



Four possible approaches were considered for national assessment approaches. These were 



The preference for the Victorian appr^^ach to profiling was expressed, but it was 
recognised that considerable work was needed before it could be made operational. In this 
piper, the potential of the Victorian approach to subject profiling will be outlined for 
general language profiling. There is of course a great deal of work to do and most of it 
may need to be done through specific subject associations or with assistance from grants 
from state and commonwealth bodies. The profiling of languages will not be a short term 
approach unless muc.n can be borrowed from other work elsewhere. It is essential however, 
that it be done. 

Remember, it is not feasible to argue th^t the national languages policy enables the 
schools and systems to argue for nore resources without the other side of the equation 
being put forward, Success in rihe development of the national language- policy does not 
only mean the provision of courses. It does not mean the implementation of programs of 
language instruction, of professional development of teachers, of the provision of 
materials, of support agencies. If it cannot be shown that these lead to the development 
of bilingualism in terms of proficiency in more than one language for all students, then 
the policy cannot be shown to have been successful* The input monitoring must be 
associated with output monitoring* There is no reason any more to believe that we can 
evaluate courses, programs and even systems in terms of the expenditure, the number of 
courses, the number of students involved* the materials used, the language laboratories 
developed and so on. The bottom line is the language proficiency of students who emerge 
from the course after exposure to the newly developed teachers, the new materials, the 
time given in the curriculum and the other resources put into the program. If all of this 
does not lead to an effective bilingual society then the program and the national language 
policy is a failure in terms of its primary objective. 

What are subject profiles? They are really methods of reporting. Within each area of 
language levels, of development need to be identified for each major component. So there 
is a need to gain agreement on the basic components of language development and to get an 
agreement on what is rnean^ by proficiency. There are numerous studies of these issues and 
surp-isingly a great deal of agreement. 

Asseisment of proficiency indicates the highest level of sustained performance of an 
individual (Byrnes and Canale, 1989), Proficiency is defined as obmerted behaviour and 
cannot be accounted for by any single unitary underlying ability. There its general 
agreement in the language literature that proficiency develops in the four so called macro 
skills of speaking, listening, reading and writing. Proficiency in speaking does not imply 
proficiency in reading or in any other language modes. In fact the discrepancy usually 
exists between language modes and the discrepancy is usually higher at the more advanced 
levels of proficiency. Galloway (1987) argues that there are four bamic areas in which 
criteria for assessing proficiency need to be addressed for each of the four macro skills. 
These are the f ^pcytion of the language being used, the content of the language, the 
context in which the language is being useu and the accuracy of use* Each of these are 
argued to affect the way in language can be demonstrated. While theme may not 
demonstrate the exclusive nature of language development and assessment it does present a 
useful framework for the assessment of language emergence and of proficiency overall. 



(i) 



statewide testing 

national testing 

expert appraisal - inspectors 

national subject profiles* 



(ii) 



(ill) 



( iv) 
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Thert is a\80 soae nevd to avoid the interpretation that the four areas of assesvient are 
also discrete. For the purposes of this discussion, it helps tc sioplify the frame of 
reference for assessaent and this can be presented as follows, 

S L R W 

Function 
Content 
Context 
Accuracy 

So if these can be considered as the basic fraae of reference then we need to identify the 
indicators of growth within each of these 12 areas. There is then a need to identify the 
levels of development such that the developnent of the «ubject profiles would need to be 
closely linked to curriculum development. It should then be possible to develop a 
framework comprising a sequence of levels through which students progress due to exposure 
to the curriculum. Some students of course will progress faster than others. Because 
the levels will refer to sequenced performance levels within a subject area they should 
not be directly related to age/grade performances of students. The performance of students 
of a particular age grade would span a number of these levels and even an individual 
student may be developing competence at several levels. 

The levels of the profiles need to be defined observable language behaviour which is 
elicited by a series of assessment tasks, not unlike the Standard Assessment Tasks (SATs) 
currently being developed by the Victorian Curriculum and Assessment Board. The 
difference would need to be the identification of the levels of development in advance of 
the standard assessment tasks. This would avoid the now apparent difficulties of 
developing the assessment tasks and then interpreting what performances on these mean in 
terms of progress or growth in the curriculum area. The assessment tasks say have 
different styles in different systems or even in different schools. They could be a mix 
of pencil and psper tests* observation of students performances against set criteria* 
assignments, interviews, practical tasks, essays, reading and role playing or simulation 
tasks and so on. Various systems may have different preferences. A student may be 
assesses to be at a hypothetical level 4 if there was evidence of being able to perform 
the tasks that define level 4 proficiency but not the tasks appropriate to level 5 
prof iciency , 

The following Diagram illustrates the potential of the system and perhaps what such a 
subject profile might look like. 
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Figure 1 

Subject Profile and Levels of Development 



The assessaents of students should be aiade by teachers in schools using the standard 
asaeosfflent tasks. The student performances can be interpreted by descripticne provided in 
each of the levels of the subject profiles. The levels and subject profiles can then be 
used to report to parents and to the ministry in turn through its reporting network which 
is being developed through its various branches. The ministry can then aggreqat? at each 
level to avoid school level compa. isons where this is seen to be unnecessary. Moderation 
would be necessary to avo;d localization of standards (Black, I'^S'^i and to ensure that 
comparability of standards is achievable. The student assessment would be criterion based 
In that the achievement of the students would be described by what they can do rather than 
what might be expected by an age/grade group or by comparisons to other students. The 
figure below illustrates the relationship between the levels and the standard assessment 
tasks . 
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LANGUAGE TASKS 



MATRIX WORKSHEET A 



TASK A 



TASK B 
TEST A 

WORKSAMPLE A 

TEST B 



MATRIX WORKSHEET B 



TASK C 



WORK SAMPLE B 
ETC 



ASSESSMENT E'iSSl^UM 
TEACH ER CONTROLLED OPTIONS VEACH/P PSflt^EP CPITgPlA 

Figure 2 

Ppyible Assessment System . Profic^pncv Levels and SATs. 

On the left of the Figure, is a collection of potential assessment strategies such as 
formal tests, performance tasks (such as reading aloud), and work samples. These 
matrices, tests and performance tasks are expected to reflect local curriculum models. A 
system of moderation, across year levels and across schools can assist in developing 
reliability of assessments and lead to considerable common interpretation of performances. 

On the right of the diagram is a symbolic representation of the pioficiency levels. The 
levels are a reporting framework which can satisfy a number of requirements. Clearly they 
can be used for descriptive reporting and profiling of individual student performance and 
they can al:io be used for aggregated reporting at a system level using a rating scale 
method such as that presented in this paper. Norm referenced interpretation is possible 
where this is considered necessary. The blurring of the boundaries between criterion 
referenced and norm referenced interpretation is a by product of the use of item response 
theory. 

There are several advantages of such a system of assessment which relies on standard 
assessment tasks and on common subject profile reporting. 

Development . 

The Victorian approach to profiling has been developed through the identification of 
reading and writing levels for students in firmt language. These have been based on the 
ASLPR and the ACTFL guidelines and the procedure umed was as follows. 



Mthod: 



The study used the following steps: 

(i) Workshops with classrooi teacher:? to define ^he observable behaviour as indicators of 
development . 

;ii) Intensive observations of students to validate, using group loderation, the 

definition of each indicator* 
:iii) Surveys to identify the neasurenent properties of the indicators and the 

devwlopnent of the initial development scale* 
( iv) Consultation with expert inforaants to modify the language development scales. 
(V) Field testing the scales; establishing rating norms* reliability and criterion 

validity estimates. 

fvi) Calibrating and anchoring the band levels with specific assessment tasks. 
Vorkmhopm 

Almost 100 teachers spent four days spread over a school year working in syndicates of six 
in structured workshops, developing their skills of analysis, observation and moderation. 
The workshops used an analytical method which combines the identification of 90«lS' the 
delineation of appropriate outco«em associated with each goal and a range of methods of 
gathering information, or evidence, that the outcomes have been achieved. The methods 
of gathering information were called amsemmment methods. These in turn were matchta t.^ 
outcomes for each goal. The evidence, which each assessment method was used to gather* was 
written into the cell of the matrix worksheet- This evidence was called the performance 
indicator. 

In a series of two day workshops, the teachers were introduced to the idea of profiling 
using a structured program, A mixture of speakers and activity sessions both informed the 
teachers of the background, developments to date and expectations of the project. Using a 
group consensus technique (Blachford, 1985), the teachers were asked to define the areas 
in which language developed within the four macro skills of reading, writing listening and 
speaking. However, these really only help to identify more specific areas of learning. 
The groups of six then became syndicates for the purposes of development and remained as a 
working group for the duration of the project. Each syndicate was asked to define the 
stages of development as outcomes of learn. ng. The teachere are asked to define the 
techniques of assessment they use. The difficult part was in cross referencing the 
results of these two sessions and creating a matrix into wh. h the performance indicators 
are written* This can take a long time and typically involves a change of thinking by the 
participant teachers as there was a need to focus on the observabl^^behaviour of the 
student and not on the interaction between the teacher and the student. 

Figure 1 shows how these aspects (areas, outcomes, assessment methods and indicators) were 
combined into a worksheet. In the workshops teachers referred to the outcomes as 
milestones and the goals were referred to as aream of literacy. These terms were retained 
for the duration of the project becai'se the teachers working in the project felt 
comfortable with the terms. 



Place Figure 3 about here 



Twenty four matrix worksheets were developed covering a range of literacy areas. An 
example is shown in Figure 4 below. It illustrates the relationship between the goal of 
I>mTmlopin9 an ayproach to Unknotm Hordm, the outcomes, such as: 

o Seeking help from others, 
o Ueing Vieual "luee, 

o Uming Auditory or grapho phonic cues, 
o Ueing Sesiantic and syntactic cues. 
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and the assetsMent B«thod« shown as. 



o Direct obstfrvsttion and anecdotal records, 
o Listening to oral reading, 
o Conferencing yith students. 



Place Figure 4 about here 



Note that the assessient methods offer the teacher a wide range of techniques and are in 
accord with the alnisterial expectations ot the asseesient method. 

Claacroo* Obsarratlon 

Notions of Ware and Cool have been devised to assist in the field trials of the workshop 
Materials. A war» teacher is ore who has attended the workshop. A warn class is the clas*" 
of the workshop teacher and a war» matrix is one developed by the teacher using it. The 
tsachers trial their own matrices in their own classroom. That is we have a warm teacher, 
a warm class and a warm matrix. Clearly it is not possible to hava a cold teacher with a 
warm matrix and this reduces the range of combinations to six. As part of the dove^ :pment 
process in workshops, four combinations were used all involving warm teachers. Later 
field trials involved cold teachers with cold matrices in their own (warm) classes and in 
other (cold) classes. 

When the teachers take the matrives away and try them out in their own classroom, this is 
a warm trial. In these trials, the teachers see if they can recognise the performance 
indicators. They check to see if the description of the assessmc'nt technique is 
appropriate. They check to see if the milestone/outcome is a realistic description of 
their students* progressive developient. Then they communicate these data to the workshs^p 
facilitators and with each other. They also gather examples of student work, where 
possible, to illustrate the performance indicator and prepare to table thio at the next 
meeting of the syndicate at the next workshop. 

At the subsequent workshop, time is devoted to discussion of the trials of the matrix in 
the warm classrooi. Teachers have been networked between workshops as well. They had 
communicated any changes they saw as being necessary. Each teccher contributed their 
experience of observation and how the children exhibited the performance indicators. They 
compared their experiences an:^ recommended revisions of the original matrix. Typical 
experience was that the matrix wac^ completely re written after the warm trial. 

The matrices were exchanged across syndicate grov ^s after the initial revision and taken 
back to the schools for use. That is the trial involved warm teachers, using cold 
matrices in warm classes. 

At a the next workshop day, the teachers again provided feedback on the use of the 
matrices in identifying appropriate student behaviour using the assessient methods in the 
matrix. This sets up the situation in wjiich all teachers can comment on the work of their 
colleaguem in developing the initial matrices and can begin to lake their own revisions of 
others* work. This was the moderation of aatrices across syndicates in that agreement had 
to be reached that the indicator could be observed using the assessment method and was 
indicative of the learning outcoae listed at the top of the matrix. 

Now all matrices become warm for the workshop teachers. That is, all teachers had had a 
hand in developing them and in revision of indicators and outcome statements. The next 
trial is a field test of all matrices by the workshop teachers using either a rating scale 
to record their observations (O»not seen;l=m&ybe; 2»yes) or i method of recording the date 
on which a specific student exhibited: the behaviour. The thinking behind this was to let 
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the observations of the students indicate the general trends in the patterns of emeraence 
of language behavior. Soie teachers also trialled the latrices out in classes cf their 
colleagues at school That is we has trials using warm teachers, war« aatrices and cold 
classes. 

Only one limited trial was conducted using cold teachers, cold matrices and warm classes. 
The rating scale approach was used but insufficient information was given to the teachers 
and the use of the matrices under these circumstances was not successful. Some alternative 
method of presenting and training the teachers needed to be developed for this approach. 
The matrices were too complex* too detailed and [presented the teachers with an 
overwhelming amount cf detail and work in assessAno and recording the behaviour of 
individual students. 

Indicatorm and Scale DevelopMent 

Instead of using the matrices* the indicators were extracted into a series of checklists. 
Again no teacher could possible observe all of the indicators with all of their students. 
There uere several hundred indicators of language development. Accordingly a series of 
overlapping sub lists were developed so that every teacher would gather information on all 
indicators but it was not necessary to gather all information on every student. This 
ensured that every student had some observational data collected and every indicator was 
observed (or not). These lists of indicators were distributed among project teachers to 
gather data for calibration purposes. A rating scale was used to show the degree to which 
each of th^se indicators was present in the reading and writing related behaviour of the 
student. A zero <0) was to be used if the teacher had not observed a student exhibiting a 
performance indicator. A one (1) was to be used if the teacher had observed th*^ behaviour 
but was not convinced that the behaviour was consistent and that this type of behaviour 
was still developing. A two (21 was to be used if the teacher considered that the 
performance indicator was now an established part of the student's repertoire of reading 
related behaviour. when the ratings were coupled with dates of observation of the 
behaviour emergence, the teachers were able to develop a short-hand way of recording their 
observation of the students' developing reading and writing skills. Teachers in 15 
schools rated 286 students on a total of 147 indicators of reading behaviour. Teachers in 
38 schools rated 578 students on 245 indicators of writing behaviour. Details of these 
analyses ^re provided by Griffin ana Jones (1988) and by Griffin (1^89). 

The Rasch Rating Scale model of the Item Response Theory (Andrich, 1^78). enabled the 
indicators to be calibrated so that all performance indicators could be mapped onto one 
continuous developmental scale. The advantage ot this methid is that both indicators and 
students can be mapped onto the same underlying growth continuum or scale. The students 
were then compared directly to indicators of general reading and writing development. 

Proficiency Levels 

The full list of indicators was examined for patterns which might be useful in summarizing 
the indicators into groups xn similar ways to the aggregation of the indicators in the 
language acquisition scales such as the ASLPR (Ingram, 1984). Several patterns were 
evident in the list of calibrated indicators of reading behaviour* The progressions 
seemed to be related to underlying factor* such as attitudinal behaviour* influence of 
reading on yriting, role playing, retelling behaviour, react ione to reading aaterials, 
analyeie and interpretation, eoeial or interactive rolee in reading behaviour, nord 
approach skille, typee of reading materiale ueed and so on. The;*e trends only helped to 
group the indictors. The labels given to them do not matter in the overall development of 
the proficiency scales. The groups of indicators were called band* and were developed in 
both reading and writing. A reading band for example, contained a description of a very 
broad range of reading behaviour rather than a discrete point of development. There were 
seven reading band* identified and nine writing band* but the nueber of bands does not 
represent anything other than the apparent grouping* of indicators. The bands were 
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labeled from A throuah G for reading and A through I for writing, setting band A at the 
earliest developnental level. The bands i: e cunulative. That is. a student placed at 
Band E was likely to have the behaviour patterns indicated by Bands A, B, C and D. 

Consultations with Export InforMnts 

The draft foras cf the reading bands were distributed to teachers and a representative 
saaple of acadenics, consultants, and inspectors and other ministry advisors in several 
Australian states, in New Zealand and in the United Kingdoa. They were asked to act as 
"expert inforaants'* and to review the draft version of the bands; to advise on the need to 
edit, delate or move the indicators included in the bands or if they considered that 
important indicators of the developaent of reading were missing, to suggest the addition 
and to recoaaend the appropriate location. Advice was also sought on the structure, 
appropriate use and suitability of the bands. 

Field Trials 

After revision by various groups of teachers and language specialists, a draft version of 
the Reading bands was prepared for field trial in 105 schools throughout Victoria. The 
writing bands were not at the same stage of development and are scheduled to be trialled 
in the large sample of schools in 1^89. A rating scale was used which described the 
teachers* observations in terms of the student exhibiting 

3, If the student has established the behaviour pattern and consistently exhibits 
all or most of the behaviour described in the band. 

2. If the student la developing tj^^ by^j^viour pattern such that some but net all ct 
the behaviour for a band is often exhibited, use a code of 2 for that band, 

1. If the student ia beoinninQ to show aigna of the behaviour pattern -f a band 
level in that only a little of the pattern is shown, use a code of 1 for that 
band . 

0 If the student ahowa nope gt th^ behaviour pattern for a band level, use a code 
of 0 for that band. 

Teachers in primary schools were asked to rate students at years 1/3 and 5, and to 
administer a standardised test* The Primary Reading Survey Test (Form AA) ACER# 1981) was 
administered to year one students and the lest of Reading Comprehension (TORCH) te<*t 
(Mossensen, Hill and Masters (1984) was administered to years 3 and 5. Secondary schools 
were asked to rate students in years 7 and 9 and to administer the TORCH test to these 
students. Item level information was provided by t>0 teachers and these data were used to 
equate the bands and the tests. The results of this analysis art reported elsewhere. 
Teachers in all 105 schools provided total test score and band ratings for students. These 
have been used to estimate the internal consistency reliability and the criterion 
validity. A small number of teachers were asked to rate their students before and after 
the school holidays in order to estimate the intra rater reliability. More than 4000 
atudenta were asaesr^d uaing the reading bands. 

The development of the proficiency bands has enabled two forms of monitoring to be 
introduced. Clearly the one-off assessments when the data is collected using the teacher 
judgement as a aeans of assessing students, is what can be called a snapshot survey. 
Where the teachers were recording the dates on which they o'- served the behaviour emerging 
it ia called a longitudinal approach. But the proficiency bands really coabine both foras 
of L^arvey. Teachers professional judgements of students work build up over a long period 
of time. In many inatances, they are informed by such assessments a* standardised tests, 
assignments, work samples, interviews, and other forms of asaessaents including student 
aelf assessaent. All of this inforaation goes into foraing the teacher *s judgeaent in a 
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"snapshot** application of the subiect profiles. It is true that teacher ^udqe^ents are 
affected by localisation of standards i Black. 1'87) and that th^re r^ter effects ^nd 
halo effects operating. However these aay be «08t serious at the level of the individual 
student and sone controJ over thei can be exercised usina a eystew cf moderation not 
unlike the system used by VCAB with the year i: assessments. This would not iapose iny new 
approaches on the secondarv teachers, but primary teachers involved in the proiect found 
the idea novel but valuable. The professional development spin-offs were obvious ar.d 
served to assist in iusues such as reliability and validity of judgements. When the 
teachers ratings were compared to standardised test results at a rlass leys'! there was 'sn 
85% consistency between the two sets cf data. 

One further thing needs to be pointed cut. The teachers involved in the pro-iect riave 
developed an ownership of the scheme. This surely is a further strength of the proiect. 
Further developments of vhe project are planned for the future. The same approach is b^ir.o 
adopted in ESL, Science, Social Education and Mathematics. There is Ministerial 
commitment to the method in victoria; there is a general interest in using the approach in 
the United Kingdom through one nf the consortia developing the national assessment for and 
there is general interest in other Australian states and in New Zealand, Expressions of 
interest have also been made by groups involved in language education, tfhat is needed ;s 
a group of teachers willing to take the lead and begin the deveiopaent of proficiency 
scales and assessment task banks for the different languages. 

Advantages . 

;i) The asses<iment system should en^.ble an analytical or diagnomtic approach to bo 
adopted in assessment. The term analytical is preferred because it does net infer that 
there are only problems. An analysis seeks both strengths and weaknesses ^nd prcvides 
information which ran be used to identify appropriate targets and paths tcr teaching and 
learning, 

(ii) The assessment system should be criterion referenced in that the student's 
performance or behaviour pattern is compared to a series of tasks. Criterion referenced 
interpretation enables the student? development to be interpreted in terms of beh^iviours 
which they can demonstrate. If the proficiency levels contain sets of irdicatcrs which 
enable criterion referenced interpretation* each student's development can be interpreted 
in terms of the descriptive profiles of language behaviour rather than ir, terms of aae or 
grade norms. 

(iii) The assessment system should enable interpretation of assessment :f learnir!9 * 
be progremmive^ deTelopmental and cumul&.tive. There is a need to trace cut a oeneral 
direction of development of students without piescribing the precise path of 'development 
for any individual student* The notion of accumulation is important. The assessment 
system needs to describe a progression of skills which are retained. It sliould neither 
describe behavicur in deficit terms nor in terms of transitory behaviours which might te 
described as stages through which students pass and leave behind. The scale needs to 
illustrate the general pattern of how skills accumulate without claiming to have 
identified all skills or to reach the definitive end of the progression. Even if an 
accumulation of skills can be defined, it is not true that the progr^^8• through this 
accumulation is always linear or monotone. Per example even the most proficient reader, 
when placed in an entirely new context, may have to employ skills which art usually 
exhibited by readers at earlier levels of development. This would not mean that the 
proficient reader's skills have diminished. The possibility of moving forward and 
backward throughout the progression cust available. Vhat is important is the idea of 

a threshold which is exhibited when the leanier is working within familiar contexts. 
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(iv) Tht asteisaent syfit^a and the Standiird Ass-ssaent Tasks should be formed usino both 
^OHMIISM soderation and espirical calibiation. If the proficiency levels are to be used 
for routine Bonitoring ot student outcomes there may be a need for a ranae of foras of 
moderation . 

(V) The assessaent tasks and curriculiM corv'jnt should be toachar controlled and 
deTeloped. This is important to the integrity cf assessment of li-arning. When the 
teacher makes the judgements about what to assess and how it should be 4S«essed, the 

information obtained is more likely to te related tc the curriculum and to be interpreted 
in the context in which the learning occurs. Externally controlled assessments cannot 
always provide this direct relevance. This is not to argue that externally developed 
standardized tests should not be used. On the contrary, the classroom teacher should be 
able to identify appropriate tests whi:h assess skills directly related to the curriculumi 
take account of the context and use the information with the general progress.or. defined 
by the proficiency levels. A strength of the criterion levelssuch as those described by 
the ASLPR is that they offer a wider frame of reierence for interpretation of test 
information than the restricted paper and pencil tasks which characterise such tests. 

(vi) The use of simple rating scales with the profiency levels and the standard 
assessment tasks also assists in making the assessment system more flexible. Graded 
assessments can be based on recording systems which only allow complete/ incomplete or 
right/wrong observations of individual tasks. However, the behaviour HA^rribp^ by each of 
the indicators may not be readily described as simply present or absent. Some language 
behaviours emerge over time and the recording process needs to allow for that if it is 
going to assist teachers in proper analyses of language proficiency development. Because 
levels such as those \n the ASLPR are criterion referenced and sequences of student 
development, they are not directly related to age or grade levels or to expectations of 
students and hence relate to learning rather than specific sub groups cf learners. 

(vii) The assessment system needs to demcistra e reliability and ¥aHdit7* Problems 
associated with rater and halo effects need to be controlled as much as possible and will 
always be present to some extent when judgement forms the overall basis of the assessment. 
Face validity can be derived from the base of ^eacher development and the 'bottom up* 
approach. 

(viii) The proposed system relies on the ability of teachers to describe behaviour of 
students in terms cowiunicable to parents. For communication to the wider community, 
distributions of levels associated with the profile levels could be used I jed on ratings, 
or estimates cf students at developmental stages for each band level. The progression 
through the levels needs to be simple to understand, and there needt to be sufficient 
levels to ensure that some progress is evident over a reasonable time* 

There are also disadvantages. 

(i) There is a need for a considerable amount of development work to be done in 
identifying the levels, specifying the criteria and establishing the bank of assessment 
tasks. This would mean that there would be considerable lead time before the approach can 
be fully implemented. However the trade off of the time and developaent effort against 
the benefits in terms of the professional development and curriculum pay offs should rot 
be underestimated. 

(ii) The system would have to be **sold** to schools and explained to t^e community- two 
guups which tend to be suspicious of new ideas and are remistant to char.ije. 

(iii) Some schools and teachers would need additional assistance* a consultancy service 
would be required. This may not be a disadvantage however, as the cross fertilisation of 
ideas and assessment materials would more than compensate for the effort involved. 
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iv) 'ronsider&ble resources would need to be developed to implement the system and then 
more to maintain it. 
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OUTCOMES /ATTRIBUTES 
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Indicator A11 
Indicator A12 


Indicator B1 








Indicator A2 j 





Indicator C2 




METHOD 3 




Indicator B3 







Performance Indicators are aligned with appropriate Outcomes and Assessment Methods 

Figure 4 

Matrix Workshftflt for Matching Qutcomesd. As sessment Methods and Indicators 
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MILESTONES 



ASSESSMENT 1 
TECHNIQUES 1 


1 ASKS OTHERS 


USES VISUAL CUES 


USES AUDITORY CUES 


USES CONTEXT CUES 










OBSERVATION 
& ANNECDOTAL 


asks adults/peers 
what a word i$. 

asks adults/peers 
what a word means. 


Eye moves between 
words and pictures. 

substitutes a similarly shaped 
word lor the unknown word. 


reuses words already 
heard in stories, wail stories 
or oral reading activities. 

uses first sound ol a word 
when attempting a new 
word. 

Attempts to sound parts of 
words. 


rereads sentence 
whenunable to read i word. 

expresses that it was 
the context that gave 
clues for the word. 

queries meaning of sentence 
whenunable to read word. 


RECORDS 


RECORDS 


asks adults/peers tor 
meaning and 
pronuncialk}nof a word. 


uses appropriate 
substitutions e.g. house/ 
home. 






PARENT/ 

CONFER. 
ENCE 


asks parent (or 
meaning and 
pronunciation 
of words. 


states the picture helped 
to read the text. 


states a word is known 
because it sounds 
right. 


rereads sentence 
when unable to read a word. 

explains (hat it was 
the context that gave 
ckJes for the word. 



Figure 4 



^ ^ Matrix Work sheet : Approach to Unknown Words 
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